ABSTRACT
OBJECTIVES: Global, COVID-driven restrictions around face-to-face interviews for healthcare student selection have forced admission staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in multiple mini-interview (MMI) methodology. This study aimed to explore test-retest reliability, acceptability and usability of the system. DESIGN, SETTING AND PARTICIPANTS: Multimethod feasibility study in Physician Associate programmes from two UK and one US university during 2019-2020. PRIMARY, SECONDARY OUTCOMES: Feasibility measures (test-retest reliability, acceptability and usability) were assessed using intraclass correlation (ICC), descriptive statistics, thematic and content analysis. METHODS: Volunteers took (T1), then repeated (T2), the automated MMI, with a 7-day interval (±2) then completed an evaluation questionnaire. Admission staff participated in focus group discussions. RESULTS: Sixty-two students and seven admission staff participated; 34 students and 4 staff from UK and 28 students and 3 staff from US universities. Good-excellent test-retest reliability was observed at two sites (US and UK2) with T1 and T2 ICC between 0.65 and 0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005 and individual site (≥0.79 p<0.001). Mean test re-test ICC across all three sites was 0.82 p<0.001 (95% CI 0.7 to 0.9). Admission staff reported potential to reduce resource costs and bias through a more objective screening tool for preselection or to replace some MMI stations in a 'hybrid model'. Maintaining human interaction through 'touch points' was considered essential. Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored. CONCLUSION: These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection.
Subject(s)
COVID-19 , Feasibility Studies , Health Occupations , Humans , Reproducibility of Results , School Admission CriteriaABSTRACT
OBJECTIVE: To evaluate effects of remote monitoring of adjuvant chemotherapy related side effects via the Advanced Symptom Management System (ASyMS) on symptom burden, quality of life, supportive care needs, anxiety, self-efficacy, and work limitations. DESIGN: Multicentre, repeated measures, parallel group, evaluator masked, stratified randomised controlled trial. SETTING: Twelve cancer centres in Austria, Greece, Norway, Republic of Ireland, and UK. PARTICIPANTS: 829 patients with non-metastatic breast cancer, colorectal cancer, Hodgkin's disease, or non-Hodgkin's lymphoma receiving first line adjuvant chemotherapy or chemotherapy for the first time in five years. INTERVENTION: Patients were randomised to ASyMS (intervention; n=415) or standard care (control; n=414) over six cycles of chemotherapy. MAIN OUTCOME MEASURES: The primary outcome was symptom burden (Memorial Symptom Assessment Scale; MSAS). Secondary outcomes were health related quality of life (Functional Assessment of Cancer Therapy-General; FACT-G), Supportive Care Needs Survey Short-Form (SCNS-SF34), State-Trait Anxiety Inventory-Revised (STAI-R), Communication and Attitudinal Self-Efficacy scale for cancer (CASE-Cancer), and work limitations questionnaire (WLQ). RESULTS: For the intervention group, symptom burden remained at pre-chemotherapy treatment levels, whereas controls reported an increase from cycle 1 onwards (least squares absolute mean difference -0.15, 95% confidence interval -0.19 to -0.12; P<0.001; Cohen's D effect size=0.5). Analysis of MSAS sub-domains indicated significant reductions in favour of ASyMS for global distress index (-0.21, -0.27 to -0.16; P<0.001), psychological symptoms (-0.16, -0.23 to -0.10; P<0.001), and physical symptoms (-0.21, -0.26 to -0.17; P<0.001). FACT-G scores were higher in the intervention group across all cycles (mean difference 4.06, 95% confidence interval 2.65 to 5.46; P<0.001), whereas mean scores for STAI-R trait (-1.15, -1.90 to -0.41; P=0.003) and STAI-R state anxiety (-1.13, -2.06 to -0.20; P=0.02) were lower. CASE-Cancer scores were higher in the intervention group (mean difference 0.81, 0.19 to 1.43; P=0.01), and most SCNS-SF34 domains were lower, including sexuality needs (-1.56, -3.11 to -0.01; P<0.05), patient care and support needs (-1.74, -3.31 to -0.16; P=0.03), and physical and daily living needs (-2.8, -5.0 to -0.6; P=0.01). Other SCNS-SF34 domains and WLQ were not significantly different. Safety of ASyMS was satisfactory. Neutropenic events were higher in the intervention group. CONCLUSIONS: Significant reduction in symptom burden supports the use of ASyMS for remote symptom monitoring in cancer care. A "medium" Cohen's effect size of 0.5 showed a sizable, positive clinical effect of ASyMS on patients' symptom experiences. Remote monitoring systems will be vital for future services, particularly with blended models of care delivery arising from the covid-19 pandemic. TRIAL REGISTRATION: Clinicaltrials.gov NCT02356081.